Machine learning (ML) classification models are increasingly being used in a wide range of applications where it is important that predictions are accompanied by uncertainties, including in climate and earth observation, medical diagnosis and bioaerosol monitoring. The output of an ML classification model is a type of categorical variable known as a nominal property in the International Vocabulary of Metrology (VIM). However, concepts related to uncertainty evaluation for nominal properties are not defined in the VIM, nor is such evaluation addressed by the Guide to the Expression of Uncertainty in Measurement (GUM). In this paper we propose a metrological conceptual uncertainty evaluation framework for nominal properties. This framework is based on probability mass functions and summary statistics thereof, and it is applicable to ML classification. We also illustrate its use in the context of two applications that exemplify the issues and have significant societal impact, namely, climate and earth observation and medical diagnosis. Our framework would enable an extension of the GUM to uncertainty for nominal properties, which would make both applicable to ML classification models.

Y et, despite this progress, the point clouds acquired by MLS systems operating in real-world environments inevitably contain uncertainty arising from various error sources during acquisition and processing. Although MLS systems have advanced rapidly in both data collection and post-processing, research on uncertainty evaluation has received comparatively less attention and remains underdeveloped (Xu et al., 2025b). From a user's perspective, the quality of point clouds from MLS systems is a critical concern. As the foundational input for many downstream tasks, inadequate assessment of MLS point clouds' quality can easily impact high-precision applications such as navigation and change analysis. This will not only undermine reliability but also result in substantial waste of time and resources, which is unacceptable in real-world applications. There is a clear need for automated and reliable solutions for uncertainty evaluation. In MLS systems, four main categories of error sources contribute to uncertainty: instrumental errors, atmospheric errors, object-and geometry-related errors, and trajectory estimation errors (Habib et al., 2009, Schenk, 2001). Considering the characteristics of these error sources, existing uncertainty evaluation methods can be broadly divided into two categories: forward modeling and backward modeling (Shi et al., 2021). The core idea of forward modeling is grounded in variance-covariance propagation, which involves detailed theoretical analysis of MLS system errors.

Background and Objective: Radiation pneumonitis (RP) is a side effect of thoracic radiation therapy. Recently, Machine learning (ML) models enhanced with radiomic and dosiomic features provide better predictions by incorporating spatial information beyond DVHs. However, to improve the clinical decision process, we propose to use uncertainty quantification (UQ) to improve the confidence in model prediction. This study evaluates the impact of post hoc UQ methods on the discriminative performance and calibration of ML models for RP prediction. Methods: This study evaluated four ML models: logistic regression (LR), support vector machines (SVM), extreme gradient boosting (XGB), and random forest (RF), using radiomic, dosiomic, and dosimetric features to predict RP. We applied UQ methods, including Patt scaling, isotonic regression, Venn-ABERS predictor, and Conformal Prediction, to quantify uncertainty. Model performance was assessed through Area Under the Receiver Operating Characteristic curve (AUROC), Area Under the Precision-Recall Curve (AUPRC), and Adaptive Calibration Error (ACE) using Leave-One-Out Cross-Validation (LOO-CV). Results: UQ methods enhanced predictive performance, particularly for high-certainty predictions, while also improving calibration. Radiomic and dosiomic features increased model accuracy but introduced calibration challenges, especially for non-linear models like XGB and RF. Performance gains from UQ methods were most noticeable at higher certainty thresholds. Conclusion: Integrating UQ into ML models with radiomic and dosiomic features improves both predictive accuracy and calibration, supporting more reliable clinical decision-making. The findings emphasize the value of UQ methods in enhancing applicability of predictive models for RP in healthcare settings.

As background to the main story it is important to understand the meaning of artificial intelligence (AI), and more specifically how its subset machine learning (ML) fits into the picture. AI can be generally defined as the theory and development of computer systems that are able to perform tasks that normally require human intelligence. As such AI systems may be adept in discovering new information, making inferences and possessing reasoning capability. ML is a subset of AI focussing on AI methods that are able to learn and adapt. AI includes symbolic computation, such as expert systems, which are not a part of ML, whereas ML builds statistical models of data that may be used for classification and prediction tasks to aid decision-making. Here we focus on ML rather than AI, but will still use the term AI when referring to the more general technology.